Designing Experiments for the Process

Most experiments are a part of a process, not an entity unto themselves, and designs that do not account for the restrictions of the process end up with inferior analyses. In this presentation the design of experiments are described as a process instead of a single entity. A design must first be approached by stepping back to evaluate how the design of the experiment fits into the whole process, identifying the process restrictions, and, finally, using the restrictions to develop an appropriate designed experiment and then the corresponding analysis. Only when the complete process involved with the design of the experiment is known can an appropriate model be constructed. Simple to complex designs have basic characteristics in common called design structures. Basic design structures are the building blocks of complex designs. The identification of the four basic structures and their restrictions is the basis of successfully identifying or classifying an experiment and of constructing the resulting analysis. This approach is a change to the paradigm for designing experiments, and the methodology is applicable to all areas of research from agriculture to manufacturing to social sciences. The following sections present a discussion of the design of experiment tools and will present several examples from agriculture, semi-conductor manufacturing, and the social sciences. INTRODUCTION The traditional methods design of experiments are taught and/or discussed in text books are not the ways design of experiments are or should be used for real world applications. Design of experiments are taught as single entities such as a completely randomized design, randomized complete block, nested design, Latin square, etc.. In reality a study will generally consist of a series of individual steps that might require the use of a series of designs. A typical experiment consists of assigning animals to the levels of a drug and then measuring one or more responses on each of the animals. The analysis of the data obtained at the end of the process is generally based on the method of assigning the animals to the levels of the drug. But, there are possibly many steps between the assignments of animals to drugs and the data one extracts at the end of the study. Ignoring what possibly happens during these steps can result in inferior quality data analysis(data with a lot of unexplained variation). One or more tools of designed experiments might be used at each of the steps of the study. For example, if blood samples are obtained from each animal, what order does one obtain the samples from the animals and then what order does one have the samples analyzed by the laboratory? Will more than one technician be involved with obtaining the blood samples? Will more than one technician be involved with the analyses of the samples in the laboratory? How long will it take the samples to be analyzed in the laboratory? Will laboratory results vary from setup to setup or day to day or session to session? When discussing design of experiments, questions like these are generally not discussed. And most surely, the consequences of the answers to those (or similar) questions are ignored. The effect of ignoring the answers to the questions is to add variability to the data set. An increase in the variability in the data set dilutes the evaluation of the means of the levels of the drug through an increase in the estimated standard error of a difference and results in increases of type I error rates. Each experiment is composed of a set of steps and the activities carried out at each step can have a big influence on the variability of the data set and unfortunately these causes often go unrecognized. It is important for the statistician, biostatistician or data analyst to understand exactly what is happening (or has happened) at each step of the process so that a decision can be made as to if one or more tools should be used to understand sources of variability in the data set and then identify and account for them by an appropriate analysis. Variability that can be accounted for by the analysis is variability that is removed from the error(s) of the model. This presentation starts out with a detailed example of a simple experiment that involves several steps with discussion as to when it may be important to incorporate a tool such as randomization, or identify which samples were processed by which technician, etc. The second part of the discussion provides a description of a set of tools available for use at any step of the experiment. The final sections provide discussions of case studies showing the effects of ignoring the intermediate steps of the process. It is important when one teaches design of experiments to inform the students that most designs consists of a set of Statistics and Data Analysis SAS Global Forum 2009